Add GStreamer AI_ML test suite by vijpatel22 · Pull Request #279 · qualcomm-linux/qcom-linux-testkit

vijpatel22 · 2026-02-07T18:47:30Z

PR – Add AI/ML GStreamer Test Suite

Overview

This pull request introduces, a self‑contained test suite for the AI/ML GStreamer suite located at:
Runner/suites/Multimedia/GStreamer/AI_ML/run.sh

The script automates the end‑to‑end validation of object‑detection and image‑classification pipelines that leverage Qualcomm‑specific GStreamer plugins.

It will:

Downloads TFLite models, sample video, and label zip‑files on‑the‑fly.
Checks that all GStreamer elements needed by the pipelines are available.
Executes both pipelines with configurable timeout, GST_DEBUG level, and output‑video path.
Handles Wayland socket discovery/adoption when a compositor is present.
Emits a result file (AI_ML.res) containing PASS, FAIL, or SKIP.
Provides detailed logging via the existing test‑kit helper functions.

🛠️ What’s Added / Modified

Runner/suites/Multimedia/GStreamer/AI_ML/run.sh - New executable script (the full implementation is attached in this PR).
Runner/suites/Multimedia/GStreamer/AI_ML/AI_ML.yaml - New YAML file
Runner/suites/Multimedia/GStreamer/AI_ML/Readme.md - New Readme file
Runner/utils/lib_gstreamer.sh - Added new functions.

smuppand · 2026-02-08T10:55:43Z

@vijpatel22 Please update the script's execute permissions and address the errors reported by shelllint.

https://github.com/qualcomm-linux/qcom-linux-testkit/actions/runs/21785074948/job/62885941654?pr=279
https://github.com/qualcomm-linux/qcom-linux-testkit/actions/runs/21785074952/job/62885941784?pr=279

vijpatel22 · 2026-02-09T08:57:23Z

Hi @smuppand, I’ve pushed the latest changes. Could you please review.

smuppand · 2026-02-10T10:26:01Z

Runner/suites/Multimedia/GSTreamer/AI_ML/run.sh

+    # Optional: make sure we have network connectivity (kept from the
+    # original script)
+    # ------------------------------------------------------------------
+    if command -v ensure_network_online >/dev/null 2>&1; then


Skip download if required assets already exist (non-empty).

Otherwise bring up Ethernet first (get_ethernet_interfaces, bringup_interface, try_dhcp_client_safe) and re-check connectivity (check_network_status / ensure_network_online).

If Ethernet fails, try Wi-Fi fallback using get_wifi_interface + get_wifi_credentials + wifi_connect_nmcli/wifi_connect_wpa_supplicant + DHCP.

If still offline/limited → SKIP (not FAIL), since downloads can’t proceed.

Hi @smuppand , thanks for the feedback. The function ensure_network_online function already implements:

First checking Ethernet interfaces (using get_ethernet_interfaces, bringup_interface, try_dhcp_client_safe/run_dhcp_client)

If Ethernet fails, trying Wi-Fi (using get_wifi_interface, get_wifi_credentials, wifi_connect_nmcli/wifi_connect_wpa_supplicant)

If both fail, returning 1 (which would result in a SKIP in the calling context)

So, it seems the current ensure_network_online function already does what the comment is asking for.

I have added the skip download part if required assets already exists, in latest commit.

I have added the skip download part if required assets already exists, in latest commit.

asset presence should be checked before forcing network bring-up. Right now the function calls ensure_network_online first, so an offline target can still SKIP even when all required assets are already cached locally. The flow should be:

first verify all required assets exist and are non-empty

only if any are missing, bring up network and download

if network still cannot be brought up, then SKIP

smuppand · 2026-02-10T10:28:01Z

Runner/suites/Multimedia/GSTreamer/AI_ML/AI_ML.yaml

+    - REPO_PATH=$PWD
+    - cd Runner/suites/Multimedia/GSTreamer/AI_ML/
+    - ./run.sh --timeout "${TIMEOUT}" --gstdebug "${GST_DEBUG_LEVEL}" --output-video "${OUTPUT_VIDEO_PATH}" || true
+    - $REPO_PATH/Runner/utils/send-to-lava.sh AI_ML.res || true


No newline at end of file.

Run.sh should handle proper exits, so there's no need to use || true for the send-to-lava.sh step.

smuppand · 2026-02-10T10:34:37Z

Runner/suites/Multimedia/GSTreamer/AI_ML/Readme.md

+
+### PASS
+- **All pipelines** successfully run to PLAYING state without critical errors
+- Output file contains `PASS AI_ML`


but your existing testkit convention is AI_ML PASS / AI_ML FAIL, and this run.sh currently prints PASS AI_ML format.

smuppand · 2026-02-10T10:34:51Z

Runner/suites/Multimedia/GSTreamer/AI_ML/Readme.md

+
+### FAIL
+- **Any pipeline** fails to reach PLAYING state or encounters critical errors
+- Output file contains `FAIL AI_ML`


smuppand · 2026-02-10T10:55:56Z

Runner/suites/Multimedia/GSTreamer/AI_ML/run.sh

+#  Source the environment and helper libraries.
+# ----------------------------------------------------------------------
+# shellcheck disable=SC1090
+. "$INIT_ENV"


init_env sourcing is wrong (double-source + idempotence variable)

smuppand · 2026-02-10T11:07:29Z

Runner/utils/lib_gstreamer.sh

+    TIMEOUT=${timeout:-60}   # default 60 seconds
+
+    console_log="${name}_console.log"
+    gst_debug_log="${name}_gst_debug.log"


README expects ./logs/..., but run_pipeline_with_logs() currently writes object_detection_console.log in CWD, not ./logs/.
Create ./logs in run.sh and pass logdir to helper, or

Make helper write under ${GST_LOG_DIR:-./logs}.

smuppand · 2026-02-10T11:08:01Z

Runner/utils/lib_gstreamer.sh

+    mkdir -p "$(dirname "${dest}")"
+
+    if command -v curl >/dev/null 2>&1; then
+        curl -fkL "${url}" -o "${dest}"


doesn’t verify download success / partial downloads

curl -fkL ... -o is okay, but you should check exit code and file is non-empty to avoid silent failures.

after download: [ -s "$dest" ] || return 1

smuppand · 2026-02-10T11:09:10Z

Runner/utils/lib_gstreamer.sh

+    cp -r "${tmp_dir}"/* "${dest_dir}/"
+
+    # Clean up the temporary folder;
+    rm -rf "${tmp_dir}" "${zip_path}"


removes the zip file always. Remove zip only on success, or gate removal behind env var.

smuppand · 2026-02-10T11:10:10Z

Runner/utils/lib_gstreamer.sh

+    fi
+
+    # ---- Get the actual size -------------------------------------------------
+    size_in_bytes=$(stat -c %s "$input_file_path" 2>/dev/null) || {


equires GNU stat -c %s

size_in_bytes=$(stat -c %s "$f" 2>/dev/null || wc -c <"$f" 2>/dev/null)

smuppand · 2026-02-10T11:10:49Z

Runner/utils/lib_gstreamer.sh

  printf '%s\n' "filesrc location=${file} ! ${dec} ! audioconvert ! audioresample ! ${sinkElem}"
  return 0
 }
+


keep mode consistent with existing utilities (usually 100755).

Signed-off-by: Vij Patel <vijpatel@qti.qualcomm.com>

Signed-off-by: root <root@lab7816.ap.qualcomm.com>

Signed-off-by: Vij Patel <vijpatel@qti.qualcomm.com>

vijpatel22 · 2026-02-17T12:24:03Z

Hi @smuppand, I’ve addressed the comments and updated the code based on the feedback. Could you please review the changes?

smuppand · 2026-03-02T16:57:38Z

@vijpatel22 Please rebase and resolve the conflicts

Signed-off-by: Vij Patel <vijpatel@qti.qualcomm.com>

vijpatel22 · 2026-03-03T18:41:43Z

Hi @smuppand, I’ve resolved the merge conflicts and updated the branch accordingly

smuppand · 2026-03-18T16:41:49Z

Runner/suites/Multimedia/GSTreamer/AI_ML/run.sh

+fi
+
+# Only source if not already loaded (idempotent)
+if [ -z "$__INIT_ENV_LOADED" ]; then


the init_env guard should use the safe unset-resistant form:
if [ -z "${__INIT_ENV_LOADED:-}" ]; then
The current direct expansion of "$__INIT_ENV_LOADED" is still not ideal.

smuppand · 2026-03-18T16:42:38Z

Runner/suites/Multimedia/GSTreamer/AI_ML/run.sh

+    # Optional: make sure we have network connectivity (kept from the
+    # original script)
+    # ------------------------------------------------------------------
+    if command -v ensure_network_online >/dev/null 2>&1; then


I have added the skip download part if required assets already exists, in latest commit.

asset presence should be checked before forcing network bring-up. Right now the function calls ensure_network_online first, so an offline target can still SKIP even when all required assets are already cached locally. The flow should be:

first verify all required assets exist and are non-empty

only if any are missing, bring up network and download

if network still cannot be brought up, then SKIP

smuppand · 2026-03-18T16:43:17Z

Runner/suites/Multimedia/GSTreamer/AI_ML/run.sh

+            fi
+            GST_DEBUG_LEVEL=$1
+            ;;
+        --output-video)


the usage/help text is stale for --output-video. It currently says the default is /test_path/video_out.mp4, but the script actually uses ./video_out.mp4 after changing into test_path.

smuppand · 2026-03-18T16:44:36Z

Runner/utils/lib_gstreamer.sh

+
+    log_info "Running ${name} (timeout=${TIMEOUT}s)"
+    # Check if timeout command is available
+    if command -v timeout >/dev/null 2>&1; then


please prefer the repo timeout helper path first and only fall back to GNU timeout if the helper is unavailable. We should avoid depending primarily on timeout for minimal Yocto images.

smuppand · 2026-03-18T16:45:38Z

Runner/suites/Multimedia/GSTreamer/AI_ML/run.sh

@@ -0,0 +1,270 @@
+#!/bin/sh
+# Copyright (c) Qualcomm Technologies, Inc. and/or its subsidiaries.
+# SPDX-License-Identifier: BSD-3-Clause-Clear


Update the license to # SPDX-License-Identifier: BSD-3-Clause

smuppand requested changes Feb 10, 2026

View reviewed changes

vijpatel22 and others added 4 commits February 13, 2026 11:39

Add GStreamer AI_ML test suite

c151334

Signed-off-by: Vij Patel <vijpatel@qti.qualcomm.com>

Fix: Add execute permission to run.sh

52d925a

Signed-off-by: Vij Patel <vijpatel@qti.qualcomm.com>

shellcheck: fix warnings in lib_gstreamer.sh

ecb24e7

Signed-off-by: Vij Patel <vijpatel@qti.qualcomm.com>

fix: keep consistent mode on lib_gstreamer.sh

6479657

Signed-off-by: root <root@lab7816.ap.qualcomm.com>

vijpatel22 force-pushed the feature/gstreamer-ai-ml branch from ce24ad8 to 6479657 Compare February 13, 2026 12:00

vijpatel22 added 4 commits February 16, 2026 00:22

fix: Add consistent .res file output to Readme.md

d6463a7

Signed-off-by: Vij Patel <vijpatel@qti.qualcomm.com>

fix: Changes to output path in AI_ML.yaml

20ed61b

Signed-off-by: Vij Patel <vijpatel@qti.qualcomm.com>

fix: Changes to run.sh revision 1

81d0af2

Signed-off-by: Vij Patel <vijpatel@qti.qualcomm.com>

fix: Changes to lib_gstreamer.sh revision 1

0c8a0f9

Signed-off-by: Vij Patel <vijpatel@qti.qualcomm.com>

Merge branch 'main' into feature/gstreamer-ai-ml

b6f6ae1

Signed-off-by: Vij Patel <vijpatel@qti.qualcomm.com>

smuppand requested changes Mar 18, 2026

View reviewed changes

Conversation

vijpatel22 commented Feb 7, 2026

PR – Add AI/ML GStreamer Test Suite

Overview

🛠️ What’s Added / Modified

Uh oh!

smuppand commented Feb 8, 2026

Uh oh!

vijpatel22 commented Feb 9, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

vijpatel22 commented Feb 17, 2026

Uh oh!

smuppand commented Mar 2, 2026

Uh oh!

vijpatel22 commented Mar 3, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants